Pinterest的技术架构
Scaling
Pinterest
Marty Weiner
Cloud Ninja
Yash Nelapati
Ascii Artist
Monday, November 11, 13
Scaling Pinterest
Evolution
Monday, November 11, 13
Scaling Pinterest
March 2010
Growth
Page views per day
Mar 2010
Jan 2011 Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
March 2010
Growth
Page views per day
Mar 2010
Jan 2011 Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
March 2010
Growth
·
RackSpace
·
1 small Web Engine
·
1 small MySQL DB
·
1 Engineer + 2 Founders
Page views per day
Mar 2010
Jan 2011 Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
March 2010
Growth
Monday, November 11, 13
Scaling Pinterest
March 2010
Growth
Monday, November 11, 13
Scaling Pinterest
January 2011
Growth
Mar 2010
Jan 2011
Jan 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
January 2011
Growth
Mar 2010
Jan 2011
Jan 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
January 2011
Growth
·
Amazon EC2 + S3 +
CloudFront
·
1 NGinX, 4 Web Engines
·
1 MySQL DB + 1 Read Slave
·
1 Task Queue + 2 Task
Processors
·
1 MongoDB
·
2 Engineers + 2 Founders
Mar 2010
Jan 2011
Jan 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
Monday, November 11, 13
Scaling Pinterest
September 2011
Growth
Mar 2010
Jan 2011
Jan 2012 May 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
September 2011
Growth
Mar 2010
Jan 2011
Jan 2012 May 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
September 2011
Growth
·
Amazon EC2 + S3 + CloudFront
·
2 NGinX, 16 Web Engines + 2 API
Engines
·
5 Functionally Sharded MySQL DB +
9 read slaves
·
4 Cassandra Nodes
·
15 Membase Nodes (3 separate
clusters)
·
8 Memcache Nodes
·
10 Redis Nodes
·
3 Task Routers + 4 Task Processors
·
4 Elastic Search Nodes
·
3 Mongo Clusters
·
3 Engineers (8 Total)
Mar 2010
Jan 2011
Jan 2012 May 2012
Page views per day
Monday, November 11, 13
Scaling Pinterest
It will fail. Keep it simple.
Monday, November 11, 13
Scaling Pinterest
If you’re the biggest user of a
technology, the challenges will
be greatly amplified
Monday, November 11, 13
Scaling Pinterest
January 2012
Growth
Monday, November 11, 13
Scaling Pinterest
April 2012
Growth
Mar 2010
Page views per day
Mar 2010
Jan 2011
Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
April 2012
Growth
Mar 2010
Page views per day
Mar 2010
Jan 2011
Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
April 2012
Growth
Mar 2010
·
Amazon EC2 + S3 + Edge Cast
·
135 Web Engines + 75 API Engines
·
10 Service Instances
·
80 MySQL DBs (m1.xlarge) + 1 slave
each
·
110 Redis Instances
·
60 Memcache Instances
·
2 Redis Task Manager + 60 Task
Processors
·
3rd party sharded Solr
Page views per day
Mar 2010
Jan 2011
Jan 2012 May 2012
Monday, November 11, 13
Scaling Pinterest
April 2012
Growth
Mar 2010
Page views per day
Mar 2010
Jan 2011
Jan 2012 May 2012
·
12 Engineers
·
1 Data Infrastructure
·
1 Ops
·
2 Mobile
·
8 Generalists
·
10 Non-Engineers
Monday, November 11, 13
Scaling Pinterest
Scaling Pinterest
Monday, November 11, 13
Scaling Pinterest
April 2013
Growth
Page views per day
April 2012
April 2013
Monday, November 11, 13
Scaling Pinterest
April 2013
Growth
Page views per day
April 2012
April 2013
Monday, November 11, 13
Scaling Pinterest
April 2013
Growth
·
Amazon EC2 + S3 + Edge Cast
·
400+ Web Engines + 400+ API
Engines
·
70+ MySQL DBs (hi.4xlarge on SSDs)
+ 1 slave each
·
100+ Redis Instances
·
230+ Memcache Instances
·
10 Redis Task Manager + 500 Task
Processors
·
65+ Engineers (130+ total)
Page views per day
April 2012
April 2013
Monday, November 11, 13
Scaling Pinterest
April 2013
Growth
·
Amazon EC2 + S3 + Edge Cast
·
400+ Web Engines + 400+ API
Engines
·
70+ MySQL DBs (hi.4xlarge on SSDs)
+ 1 slave each
·
100+ Redis Instances
·
230+ Memcache Instances
·
10 Redis Task Manager + 500 Task
Processors
·
65+ Engineers (130+ total)
Page views per day
April 2012
April 2013
·
8 services (80 instances)
·
Sharded Solr
·
20 HBase
·
12 Kafka + Azkabhan
·
8 Zookeeper Instances
·
12 Varnish
Monday, November 11, 13
Scaling Pinterest
April 2013
Growth
Page views per day
April 2012
April 2013
·
65+ Engineers
·
7 Data Infrastructure + Science
·
7 Search and Discovery
·
9 Business and Platform
·
6 Spam, Abuse, Security
·
9 Web
·
9 Mobile
·
2 growth
·
10 Infrastructure
·
6 Ops
·
65+ Non-Engineers
Monday, November 11, 13